A great deal of effort has recently been focused on developing electronic circuits which emulate higher-order brain functions such as memory learning and/or perception/recognition.
One class of circuit devices which sense an input event and output a pattern of signals which identifies that event are association networks. Association networks generally take the form of a matrix comprised of horizontal lines crossing and contacting an equally sized set of vertical lines. The horizontal lines simulate the functions of axons in the cortex of the brain and are used as inputs. The vertical lines simulate the function of dendrites extending from neurons. Each vertical line terminates at a voltage summing device which acts to simulate the function of the neuron cell body. An example of such an associative network is found in co-pending application entitled "Semiconductor Cell For Neural Network Employing A Four-Quadrant Multiplier", Ser. No. 07/283,553, now U.S. Pat. No. 4,950,917 filed Dec. 9, 1988, which is assigned to the assignee of the present application.
Within an associative network, neural synapses are simulated by circuit cells which provide electrical connection between the horizontal and vertical lines of the network. Individual synapses provide a weighted electrical connection between an input an a voltage summing element, i.e., a neuron.
These synapse cells may be either analog or digital in nature. For an analog implementation, the weighted sum of input signals is usually computed by summing analog currents or charge packets. For a general description of an associative network processing unit consisting of analog connection elements, see "VLSI for Artificial Intelligence", edited by Jose G. DelGado-Frias and Will R. Moore, Kluwer Academic Publishers, 1989, pp. 230-233.
One of the most difficult and critical tasks faced by researchers in the field of neural networks is the integration of the synapse cells, also referred to as contact structures. The several realizations that have been proposed range from non-programmable binary to programmable analog interconnections.
In an analog synapse cell, considerations of cell size and resolution of the connection weight must be carefully balanced. Furthermore, learning within an associative network requires adaptive weight values since a typical network system cycles through a series of weight changes until the entire network converges to a certain pattern which depends upon the pattern of inputs applied. Several synapse cell circuits are described in co-pending applications "EXCLUSIVE-OR Cell For Neural Network Cell And The Like", Ser. No. 309,247, now U.S. Pat. No. 4,904,881 filed Feb. 10, 1989; and "EXCLUSIVE-OR Cell For Pattern Matching And Employing Floating-Gate Devices", Ser. No. 325,380, now U.S. Pat. No. 4,999,525 filed Mar. 17, 1989, both of which are assigned to the assignee of the present application.
FIG. 2 of U.S. Pat. No. 4,802,103 of Faggin et al., discloses a contact structure which utilizes a floating-gate transistor 34. Device 34 is used to discharge a target line of the network in proportion to the amount of charge stored on the floating gate member of device 34. The magnitude of the convergence response of the network is altered by incrementally erasing the floating gate transistors. In other words, the connection strength is increased to increase the discharge current associated with the target line. A detecting circuit indicates a convergence response once a predetermined amount of charge is removed from the target line.
The chief drawback of the contact structure of Faggin's FIG. 2 is that it operates as a simple one-quadrant device. That is, Faggin's synapse cell only produces a positive activation function, corresponding to an activated excitatory connection. It is well understood that biological memories accommodate both excitatory and inhibitory connections--thus providing both positive and negative responses. Electrically, such a synapse presents either a positive or a negative resistance between an axon (i.e., input) and a dendrite (i.e., output summing line) coupled to a neuron body. A cell providing both excitatory and inhibitory connections would more closely resemble the actual function performed by a synapse within the human brain. Moreover, such a cell would have the potential to learn quicker, thereby providing faster convergence within an associative network. Therefore, what is needed is an integrated four-quadrant synapse cell which can produce both positive and negative responses.
Certain types of synapse cells do implement four-quadrant multiplication of an input pattern and a stored weight. However, these cells suffer from several important drawbacks. First of all, previous approaches generally involve numerous devices requiring large silicon areas. In many cases, they employ p-channel devices which necessitate the formation of extra n-well regions. This further increases the silicon area consumed. In addition, conventional four-quadrant synapse cells typically require the use of differential output summing lines, also referred to as column lines or bit lines, which again contribute to the complexity and size of the network.
As will be seen, the present invention covers a compact and elegant synapse cell employing a pair of floating-gate transistors coupled to a single output summing line. The invention offers the advantage of four-quadrant performance while obviating the complex synapse circuitry and differential bit lines normally associated with multi-quadrant devices. Importantly, the cell of the present invention achieves very high densities while still providing full incremental learning capabilities.
Other prior art known to Applicant includes: U.S. Pat. No. 4,760,437 of Denker et al.; U.S. Pat. No. 4,660,166 of Hopfield; U.S. Pat. No. 4,782,460 of Spencer; "Programmable Analog Synapses For Micro Electronic Neural Networks Using A Hybrid Digital-Analog Approach", by F. J. Mack et al., IEEE International Conference on Neural Networks, July 24-27, San Diego, Calif.; and "A Pipelined Associative Memory Implemented In VLSI", by Clark et al., IEEE Journal of Solid-State Circuits, Vol 24, No. 1, pp. 28-34, February 1989.